IMPLICIT ITERATION METHODS IN HILBERT SCALES 
UNDER GENERAL SMOOTHNESS CONDITIONS 



QINIAN JIN AND ULRICH TAUTENHAHN 

Abstract. For solving linear ill-posed problems regularization methods are 
required when the right hand side is with some noise. In the present paper reg- 
ularized solutions are obtained by implicit iteration methods in Hilbert scales. 
By exploiting operator monotonicity of certain functions and interpolation tech- 
niques in variable Hilbert scales, we study these methods under general smooth- 
ness conditions. Order optimal error bounds are given in case the regularization 
parameter is chosen either a priori or a posteriori by the discrepancy principle. 
For realizing the discrepancy principle, some fast algorithm is proposed which 
is based on Newton's method applied to some properly transformed equations. 



1. Introduction 

In this paper we are interested in solving ill-posed problems 

Ax = y, (1.1) 

where A G C(X, Y) is a linear, injective and bounded operator with non-closed 
range TZ(A) and X,Y are Hilbert spaces with corresponding inner products (-, ■) 
and norms || ■ ||. Throughout we assume that y G TZ{A) so that ( 11. ip has a unique 
solution x^ G X. We further assume that y is unknown and y s G Y is the available 
noisy right hand side with 

\\y-y 5 \\<5. 

In recent literature many aspects of treating ill-posed problems with noisy right 
hand side have been studied. For an overview, see, e.g., the textbooks jU [39] . 
The numerical treatment of ill-posed problems (II. ip with noisy data y 5 requires 
the application of special regularization methods. In this paper we study implicit 
iteration methods in Hilbert scales, in which regularized solutions x 5 n are obtained 

by 



X i. — X 



fc-1 



(A*A + a k B 2s )- 1 A*(Ax 5 k _ 1 -y 5 ), k = 1, 2, n, x 5 = x (1.2) 

where B : T>(B) C X — > X is some unbounded densely defined self-adjoint strictly 
positive definite operator, a k > are properly chosen real numbers, s is some 
generally nonnegative number that controls the smoothness to be introduced into 
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the regular izat ion procedure and Xq is some properly chosen starting value. In 
these regularization methods, the positive number 



plays the role of the regularization parameter. For results on convergence rates of 
this method in the special case s = we recommend the paper [7J, and for some 
extensions to the nonlinear case we recommend H3] and dU [3T] . 

Method (II. 2p with n — 1 and x = is the method of Tikhonov regularization 
in Hilbert scales which has been studied by Natterer [28]. From this paper we 
know that under the assumptions ||-B _a x|| ~ \\Ax\\ and ||5 p a^|| < E the Tikhonov 
regularized solution x s a of problem (jl.2p with n = 1, Xq = and a.\ = a guarantees 
order optimal error bounds 



(i) to the case of choosing a a posteriori by the discrepancy principle, see, 



(ii) to a general regularization scheme, see, e.g., |4"1 [35]. 

(iii) to the case of general source conditions including infinitely smoothing op- 
erators A, see, e. g., pU [T7J HQ [22j [25], 

(iv) to the case of nonlinear ill-posed problems, see, e. g., [H [121 EE EH 133], [36] . 
Our paper is organized as follows. In Section 2 we collect some preliminaries on 

properties of the implicit iteration methods in Hilbert scales, formulate our general 
smoothness conditions and give some consequences that follow from the general 
smoothness conditions by exploiting either operator monotonicity or interpolation 
in variable Hilbert scales. In Section 3 we treat the case of a priori parameter 
choice of the regularization parameter a n and Section 4 treats the case of choosing 
o~ n a posteriori by the discrepancy principle. In Section 5 we discuss practical 
issues of choosing the starting value Xq and the parameters s, n and a^, k = 
l,...,n. In particular, some fast globally convergent algorithm for realizing the 
discrepancy principle is proposed which is based Newton's method applied to some 
properly transformed equations. For testing the algorithm, numerical experiments 
are performed in Section 6. 



2.1. Properties of the regularization method. In our further study, instead 
with B, we will work with the inverse G = B~ x . Following proposition gives us 
some equivalent representation for x 5 n defined by (11.21) along with some preliminary 
properties which will be useful for deriving order optimal error bounds. 

Proposition 2.1. Let T = AG S , G = B' 1 and a n be defined by / TO|) . Then, the 
regularized solution x s n defined by U.ty) possesses the representation 




(1.3) 




e.g., [291 [331 132 E5 



2. Preliminaries 




(2.1) 
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In addition, the function g n : (0, c] — > (0, oo) with c = \\T\\ 2 and the corresponding 
residual function r n (X) := 1 — Xg n (X) obey the properties 

(i) 9nW < o- n , (2.2) 

(ii) A# n (A) < 1, (2.3) 

(iii) Ar n (A) < a~\ (2.4) 

(iv) r n (A) < a- l g n {\). (2.5) 

Proof. The proof of the representation (12. ip is standard. For the proof of (i) we 
follow the paper [7J and observe that the function r n (A) = rife=i a £:/(A + ®k) is 
monotonically decreasing and convex with r n (0) = 1. From these properties we 
conclude that 

r„(A) >r n (0) + r;(0)A. 

Since r n (0) = 1 and r' n (0) = —a n we obtain r n (A) > 1 — Acr n , which is equivalent to 
(i). The proof of (ii) follows from the representation Ag n (A) = 1 — YYk=i a kl '(A+c^). 
For the proof of (iv), we multiply (12. 5p by \o~ n /r n (\) and obtain the equivalent 
inequality 



1. 



This inequality, however, always holds true since the left hand side is the first 
order term of the polynomial in A on the right hand side. For the proof of (iii) 
we use (iv) and (ii) and obtain r n (A) < cr~ 1 g n (\) < a,^ 1 /A, which is equivalent to 
(iii)- □ 

Remark 2.2. Note that in our forthcoming analysis we will also exploit the fact 
that r n (A) < 1 for A G (0, c] which is a consequence of the nonnegativity of 
g n (A). Further note that property (iii) of the above proposition tells us that the 
regularization method (12. ip has at least a qualification of po = 1. For the concept 
of qualification, see [39J. Finally we note that our analysis does not require the 
full strength of the properties (i) - (iii) of Proposition 12.11 Indeed, property (i) 
will be exploited for the A - range A < cr" 1 , and properties (ii) and (iii) will be 
exploited for the A - range A > er" 1 . 

For deriving order optimal error bounds for — x*\\ with x s n defined by (12. ip 
we introduce the regularized solution x n with exact data by 

x n -x = G s g n (T*T)T*(y - Ax ). 

It can easily be checked that the following error representations 

x 6 n -x n = G s g n (T*T)T*(y 5 -y) and x^ - x n = G s r n (T*T)G- s (x^ x ) (2.6) 
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are valid. From these error representations we see that in the \\Ax\\ - norm and 



in the X p - norm \\x\\ p := \\G p x\\ we have 



\\Ax 5 n -Ax n \\ = \\Tg n (T*T)T*(y 5 -y)\\, (2.7) 
\\Ax^-Ax n \\ = \\Tr n (T*T)G' s (x^ - x )\\, (2.8) 
\\x s n -x n \\ p = \\G s ^g n (T*T)T*(y s -y)\\, (2.9) 



\\x^-x n \\ p = \\G s - p r n (T*T)G- s (xi - x )\\. (2.10) 

2.2. Smoothness assumptions. We formulate our smoothness assumptions in 
terms of some densely defined unbounded selfadjoint strictly positive operator 
B : X — > X. We introduce a Hilbert scale (X r ) rg R induced by the operator B 
which is the completion of C\kg&D(B k ) with respect to the Hilbert space norm 
||x|| r = ||-B r x||, r G M. For technical reasons, instead of B we will work with the 
inverse G := B' 1 , which is a bounded linear injective and selfadjoint operator 
with non-closed range 71(G). Note that the above Hilbert space norm || • || r may 
be represented by 

||x|| r = \\B r x\\ = \\G- r x\\, reR. 

In addition, according to [TU| [T9] we call a function g : (0, a] — > (0, b] an index 
function if it is continuous and strictly increasing with g(0+) = and assume 

Assumption Al. There exist constants M > m > and some index function 
q : (0, a] — > (0, b] with a = \\G\\ and b = g(a) such that 

(i) m \\q(G) x\\ < \\Ax\\ for all x G X, 

(ii) \\Ax\\ < M \\g(G)x\\ for all x G X . 

Assumption A2. For some positive constants E and p we assume the solution 
smoothness x' — Xq — G p v with v G X and \\v\\ < E. That is, 

x 1 G M PjE = {xeX\\\x- x \\ p < £"} . 

Assumption Al characterizes the smoothing properties of the operator A rela- 
tive to the operator G and allows the study of problems with finitely and infinitely 
smoothing operators A in a unique manner. Typical index functions in applica- 
tions are power type index functions g(t) = t a for problems (11. ip with finitely 
smoothing operators A and index functions g(t) = exp(t~ a ) where the inverse g^ 1 
is of logarithmic type for problems with infinitely smoothing operators A. Such 
problems appear, e.g., in inverse heat conduction. Assumption A2 characterizes 
the smoothness of the element x^ — Xq in the Hilbert scale (X r ) re ^. By using 
Assumption A2 we can study different smoothness situations for x^ — Xq. 

Let us give some comment on order optimal convergence rates for identifying x' 
from noisy data /eY under the link assumption Al and the smoothness assump- 
tion A2. Let 1Z : Y — )■ X be an arbitrary method and IZy 5 be an approximate 
solution for x*. Then, the quantity 

A(8,K) = sup{||7V \ y s G Y, \\y - y s \\ < 5, J G M p , E } 
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is called worst case error of the method 1Z on the set M Pt E- An optimal method 
lZ opt is characterized by A (5, 7Z opt ) = inf^ A (5, TZ), and this quantity is called 
best possible worst case error on the set M p ^- Under Assumption Al(ii) the best 
possible worst case error can be estimated from below by 

M^{8,n)>E[^ x (^)J with Vfc(*):=iM*) (2-H) 

provided 5 /(ME) is an element of the spectrum of the operator ip p (G), see [37] 
and [251 proof of Theorem 2.2]. This lower bound will serve us as benchmark for 
the best possible accuracy for identifying x^ from noisy data y 5 G Y under the 
link assumption Al and the smoothness assumption A2. 

2.3. Exploiting operator monotonicity. Operator monotonicity has been ap- 
plied before in different papers for deriving order optimal error bounds in regular- 
ization under general smoothness assumptions, see, e. g., [3l|2TJ[27]. In this section 
we are going to derive some consequences of Assumption Al by using operator 
monotonicity of certain functions. Considering the error representations (12. 8p - 
( I2.10p we see that two types of estimates are helpful for deriving error bounds for 
the noise amplification error and the regularization error in the ||Ar|| - norm and 
the H^llp - norm, namely estimates of the type 

||/i(T*T)G-2;|| < ||Cr p x|| and ||G s - p x|| < ||/ 2 (T*7>|| 

with certain functions fi, f 2 and some constant p > from Assumption A2. We 
will derive such estimates from Assumption Al by using the concept of operator 
monotone functions which is based on the concept of semiordering. Note that 
for two nonnegative, self-adjoint bounded linear operators Si, S 2 £ C(X) the 
semiordering Si < S 2 is defined by (S\x,x) < (S 2 x,x) for all x G X, or equiva- 
lently, by || S'^ 2 x || < ||5 , 2^ 2x ll f° r an x G X. 

Definition 2.3. An index function / : (0, a] — > M is called operator monotone 
if and only if for any pair of self-adjoint linear operators S±, S 2 with spectra in 
(0,a], the relation Si < 5*2 implies the relation /(Si) < /(S2). 

Properties and examples for operator monotone functions may be found in 
[21 |2U1 EH]- Our further study is based on several functions. The first function is 

Vv(A) = A r ^(A), VV : (0, a] (0, a r g(a)} (2.12) 

with g from Assumption Al, a = \\G\\ and arbitrary constant r for which A r £>(A) 
is monotonically increasing. Two other functions h and w are 

h(t) = U s " 1 (v / t)j P *, w(t) = l/h(t) (2.13) 
with constant s from f)2.ip and p from Assumption A2. 

Remark 2.4. The function h defined by (I2.13P possesses the following properties: 

(i) Due to the identity \fih{t) = ip p ((^i?) -1 (i)) , the function t — > y/ih(t) is an 
index function and hence monotonically increasing in the both cases s > p 
and s < p. 
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(ii) In the case of high order regularization with s > p, the function h is 
non-increasing. Hence, for s > p, the function t — > h(t)/y/t is always 
monotonically decreasing. 

In our next proposition we derive some estimates by using Assumption Al. 

Proposition 2.5. Let h and w be defined by \2. Then, 

\\w(T*T/M 2 )G~ s x\\ < \\G~ p x\\ under Al(ii) and w 2 operator monotone, (2.14) 

|| w(T*T/m 2 )G~ s x || < ||G _p x|| under Al(i) and h 2 operator monotone, (2.15) 

||G s_p x|| < \\w(T*T/m 2 )x\\ under Al(i) and w 2 operator monotone, (2.16) 

||C s_p x|| < \\w{T*T / M 2 )x\\ under Al(ii) and h 2 operator monotone. (2.17) 
Proof. First, let us prove (12.141) . It follows from Assumption Al (ii) that 
\\AG s x\\ = \\Tx\\ = \\{T*T) l ' 2 x\\ < M\\g{G)G s x\\, 

which may be written in the equivalent form T*T/M 2 < g 2 (G)G 2s . By using the 
function ip s defined by (I2.12p . this estimate can be written as T*T/M 2 < ip 2 {G). 
Since w 2 := 1/h 2 is assumed to be operator monotone and since w 2 (ip 2 (G)) = 
G 2s - 2p we obtain that w 2 (T*T/M 2 ) < G 2s - 2p which gives \\w (T*T/M 2 ) x\\ < 
\\G s ~ p x\\ and hence (I2.14p . Second, let us prove (I2.15p . The link condition Al(i) 
may be written as ip 2 (G) < T*T/m 2 . Since h 2 is assumed to be operator monotone 
and since h 2 (tp 2 (G)) = G 2p - 2s we obtain that G 2p ~ 2s < h 2 (T*T/m 2 ). Since 
t — > — 1/t is operator monotone there follows that w 2 (T*T/m 2 ) < G 2s ~ 2p , which 
gives ( I2.15p . The proof of the estimates (I2.16P and ( I2.17P is similar. □ 

Example 2.6. (Finitely smoothing case). Let us assume that the operators A* A 
and G are related by 

A* A = G 2a (2.18) 

where a is some positive constant. In this case both Assumptions Al(i) and Al(ii) 
hold true as equality with £>(A) = A a , m = 1 and M = 1. We easily see that the 
function g is an index function and that the function ip s defined in (12 . 1 2[) attains 
the form ip s (X) = X a+S . Since il)~ l (^/i) = ^/( 2 «+ 2s ) we obtain that the functions h 
and w defined in ( I2.13P possess the representations 

h[t) = t^^ , w(t) = t^+^ . 

Power functions t v are operator monotone for < v < 1, see [20] . Hence, under 
the natural side conditions p > 0, a > and s > —a we obtain 

(i) that w 2 is an operator monotone function for s > p, 

(ii) that h 2 is an operator monotone function for s < p < 2s + a. 

2.4. Interpolation in variable Hilbert scales. By interpolation in variable 
Hilbert scales we can estimate the intermediate norm ||x|| if estimates for some 
weaker norm ||^(G)a;|| and some stronger norm \\x\\ r are known. Variable Hilbert 
scale inequalities have been introduced by Hegland, see [9j [10]. Such inequali- 
ties which extend the classical interpolation inequality became a powerful tool 
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in the analysis of regularization under general smoothness conditions, see, e.g., 
[TBI [T9l 1251 1261 1321 [37] . Variable Hilbert scale interpolation is sometimes also 
called interpolation with a function parameter, see [U [23] . In our paper we are 
aiming to combine special variable Hilbert scale inequalities with tools from oper- 
ator monotonicity. 

Proposition 2.7. Assume r > 0, \\x\\ r < C\ and \\g(G)x\\ < C2 with some index 
function g and constants C\, Cq,. Let £ r (i) := ip 2 ^ 1 ^ 2 ^) be convex where ip r is 
given by $2.12\) . Then, 

' 'c 2 



\x\\ < C\ 



Cl 



(2.19) 



Proof. Let E\ the spectral family of G 2r . Since £ r is convex we may employ 
Jensen's inequality and obtain 

^ /^(A^Adll^xll 2 _ \\g(G)x\\ 2 



£r 



X 



X 



< 



x 



x 



or equivalent ly, 



\x\ 



x 



increasing we obtain that t — > tijj r (l/t ' r ) is decreasing. Hence 



I l/r 
! r 

l/r\ 



< \\g(G)x\\. Since g(t) = t r ip r (t) is 



Ci • lf) r 



Ml 



l/r 



l/r 



< llxl 



\X 



I l/r 



\X 



I l/r 



< ||^(G)x|| < C2- 



Rearranging terms gives (I2.19p . 



□ 



In our next proposition we provide a further estimate which is based on inter- 
polation arguments. 

Proposition 2.8. Let x n be the regularized solution Ii2. 1\) with exact data y, let 
g be an arbitrary index function, let Assumption A2 hold and assume < s < p. 
Let in addition ip s be defined by Ii2.12\) and 

f{t) ■= $ (tV(2 P -2 S )) ( 2 20 ) 

be convex. Then, for all regularization parameters a n , 



X^ ^2s— p — E 



1>: 



-1 



\g{G){x r 



x 1 



E 



2p-2s 



(2.21) 



Proof. Let us introduce the abbreviation z 



x 



t _ 



identity G~ s z = r n {T*T)G~ 



[x 



t_ 



x n . From (I2.6P we have the 



,1/2 



Xq), and due to r n (A) < 1 we have the estimate 



(T*T) || < 1. We use these properties and obtain due to Cauchy Schwarz 



inequality and Assumption A2 that 



= \\r n (T*T)G- s (x^ -x )\\ 2 
< \\r l J 2 (T*T)G- s ^-x )\\ 2 
= (G p - 2s z,G- p (x^ -x Q )) 



< E\\z\ 



2s-p- 



(2.22) 
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In the case s = p, ( 12.211) follows from (12.221) . In the case < s < p, our next 
aim consists in deriving a second estimate that relates the intermediate norm 
ll^lhs-p with the weaker norm ||^(G)z|| and the stronger norm \\z\\ s . We derive 
this estimate by interpolation in variable Hilbert scales. Since / is convex we may 
employ Jensen's inequality and have 

UHl-p\ _ . f J \ s ' p ■ X s d\\E x z\\ 2 \ ff(\ s ~P)-\ s d\\E x z\\ 2 



\4s J \ / A s d||^A^|| 2 J- f\ s d\\E x z\\ 2 
where E x is the spectral family of G~ 2 . Since f(\ s ~ p )X s = f? 2 (A -1 / 2 ) we obtain 
f n\4l- P \ < jQ 2 (X-^)d\\E x z\\ 2 = \\g(G)z\\ 2 _ (2 _ 23) 

V \\ z \\a J \\ Z \\'s \\ Z \\s 

Now, let us eliminate \\z\\ s in estimate ( 12.23D . We write estimate ( 12.22ft in the 
equivalent form 

\\A\y s 2 - r JE l/2 < \\zhs- P l\\z\\ s (2.24) 
and introduce two auxiliary functions g and r by 

g{t) := t~ 2 f(t 2 ) and r(t) := tf(t) = (t 1 /^' 2 ^) _ (2>25) 

Since / is convex and /(0) = 0, g is monotonically increasing. Hence, by ( 12. 24ft . 
the monotonicity of g and (12. 23ft . 

Multiplying by ||z|| 2 /i? 2 gives 



^Ihs-p V \\ z \\s J |p|l2s— p 



\z\\ 2s - p \ \\q(G)z 



12 



E J ~ E 2 

Since the inverse r~ l has the form r~ x (A) = 1 (■\/X)] 2p ~ 2s 7 we obtain ( 12.21ft . □ 



3. A priori parameter choice 

In this section we make use of Proposition 12.51 for estimating the total error 
in different norms in case the regularization parameter o n from (II. 3p is chosen a 
priori by 

2(s-p) 



<7„ 



-1 



E 2 



S 



mE 



(3.1) 



We note that in the finitely smoothing case of Example 12. 6l the a priori parameter 



choice (13. ip attains the form a n 1 



(m_e) 



2(s+a)/(a+p) 
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3.1. Error bounds in the \\Ax\\ — norm. We start by providing error bounds 
for arbitrary a n > 0. 

Proposition 3.1. Let x s n be defined by \2. h and w be defined by and 
assume the solution smoothness A2. 

(i) High order regularization (s > p): If w 2 := 1/h 2 is operator monotone, 
then under the link condition Al(ii), 

\\Ax s n -Ax<\\ Kd + E^h^/M 2 ). (3.2) 

(ii) Low order regularization (s < p): If h 2 is operator monotone and if 
h(t)/\/t is decreasing, then under the link condition Al(i), 

\\Ax s n - Af\\ < 5 + E^h^/m 2 ). (3.3) 

Proof. For estimating \\Ax & n — Ax n \\ we use the error representation (12.71) and 
obtain due to Ag n (A) < 1, see ( I2.3p . the estimate 

\\Ax 5 n -Ax n \\ = \\T g n (T*T)T*(y s -y)\\< 5 sup \Xg n (X)\< 6. (3.4) 

A 

For estimating ||Ar n — Ax^\\ in the high order case s > p we use the error represen- 
tation f !2.8p . exploit Assumption A2 and estimate (I2.14p which requires operator 
monotonicity of w 2 and the second link condition Al(ii) and obtain 

\\Ax n -A X m = \\Tr n (T*T)G- s (x ] - x Q )\\ < \\Tr n {T*T)h (T*T/M 2 )\\ -E (3.5) 

For estimating the norm term in (13.51) we distinguish two cases A < a^ 1 and 
A > cr~ l . In the first case A < a^ 1 we use g n (\) > 0, or equivalently r n (A) < 1, 
exploit the increasing behavior of yth{t) that holds true due to Remark 12.41 (i) 
and obtain 

V\r n {X)h{X/M 2 ) < V\h(\/M 2 ) < ^fo^hip^jM 2 ). 

In the second case A > a^ 1 we use Ar„(A) < cr^ 1 , see ( 12. 4p . exploit the decreasing 
behavior of h(t)/\rt that holds true due to Remark [231 (ii) and obtain 

V\r n (X)h(X/M 2 ) < o-- l h(\/M 2 )/V\< y^hia^/M 2 ). 

From the both cases we obtain that (I3.5P attains the form 

\\Ax n - Ax^\\ < E^fo^h{o~ l /M 2 ). 

From this estimate and (13.41) we obtain (13. 2p . For the proof of part (ii) we proceed 
analogously by exploiting ( 12. 151) instead of (12.14p . □ 

Remark 3.2. Let us discuss the monotonicity condition in part (ii) of Proposition 
13.11 for the finitely smoothing case of Example 12.61 For this example we have 

. — p — Is — a 

h(t)/Vt = t^+^y. 

Hence, h{t)/y/t is decreasing for p < 2s + a. This coincides with Natterer's side 
condition in (II. 4p . 
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Corollary 3.3. Let be satisfied the assumptions of Proposition [X71 and let a n be 

chosen by [3. Then, in the both cases (i) and (ii) of Proposition [X71 we have 

(i) \\Ax 5 n - Ax^\\ < (1+ M/m) 8, (3.6) 

(ii) \\Ax 8 n -Ax^\\ <25. (3.7) 

Proof. Let us prove estimate (13. 6p for the high order case (i). The a priori param- 
eter choice (13. ip can be written in the equivalent form 

E^ = 5w(a- 1 /m 2 ). (3.8) 

Hence, by (J3Z1 and (j3T8) . 

WAxl-Ax^W a + ^VMX^/m 2 ). (3.9) 
Since o~ l /M 2 < a^/m 2 and since \fth(t) is increasing we have 

Hcr-'/M 2 ) < -Ha-'/m 2 ). 
m 

From this estimate and (13. 9p we obtain (13. 6p . The proof for the estimate (13. 7p for 
the low order case (ii) is analogous. □ 

3.2. Error bounds in X p . We start by providing error bounds with respect to 
the || • ||p - norm for arbitrary a n > 0. 

Proposition 3.4. Let x s n be defined by \2. 1\) . h and w be defined by Ii2.13\) and 

assume the link condition Al and the solution smoothness A2. 

(i) High order regularization (s > p): If w 2 := 1/h 2 is operator monotone, 

||4 - < S^nwia^/m 2 ) + E ■ M/m. (3.10) 

(ii) Low order regularization (s < p): If h 2 is operator monotone and if 
h{t)/y/i is decreasing, 

114 - x] \\p < Sy/b^wia^/M 2 ) + E ■ M/m. (3.11) 

Proof. Let us consider the high order case (i). For estimating \\x^ — x n \\ p we use 
the error representation (12. 9p and obtain due to the estimate ( I2.16P of Proposition 
12.51 the estimate 



x s n -x n \\ p =\\G s ~ p g n (T*T)T*(y s -y)\\<5 



w [ \t*T ) g n (T*T)T* 



m 2 



. (3.12) 



For estimating the norm term in (13. 12[) we distinguish two cases A < a^ 1 and 
A > cr" 1 . In the first case A < a^ 1 we use g n (X) < o~ n , see ( I2.2p . exploit the 
increasing behavior of \/iw(t) that follows since due to Remark 12.41 (ii) the function 
h(t)/y/t is decreasing and obtain 

w(\/m 2 )g n (\)\/~\ < w{X/m 2 )\/~Xa n < w{o~ x /m 2 )y/on. 

In the second case A > a^ 1 we use Ag n (A) < 1, exploit the decreasing behavior of 
w(t)/\/i that follows since due to Remark 12.41 (i) the function y/th(t) is increasing 
and obtain 

w{\/m 2 )g n (\)V\ < w{X/m 2 )/V\ < w{a~ l /m 2 )y/b~ n . 
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From the both cases we obtain that (13. 121) attains the form 

\\ x l - x n\\ P < Sy/onwia^/m 2 ). (3.13) 

For estimating; II the high order case (i) we use the error representation 

(j2.10p . exploit the estimate (I2.14p of Proposition 12. 5} use in addition the estimate 
(I2.16P and obtain due to \\x* — Xq\\ p < E and g n {X) > 0, or equivalently r n (A) < 1, 
the estimate 

Wx^-XnWp = \\G s - p r n (T*T)G- s (x^ -x )\\ 

< \\w (T*T/m 2 ) r n (T*T)G- s (x ] - x )\\ 

< E \\w (T*T/m 2 ) r n (T*T)h (T*T/M 2 ) \\ 

< Esup\h(X/M 2 )w(X/m 2 )\. (3.14) 

A 

Due to Remark 12.41 (i) the function y/th(t) is increasing, or equivalently, w(t)/\/t 
is decreasing. Hence, from X/M 2 < X/m 2 we have w(X/m 2 ) < ^w(X/M 2 ), and 
(I3.14p attains the form 

\\x ] - x n \\ p < E ■ M/m. (3.15) 

From (I3.13P and ( 13 . 1 5[) we obtain (I3.10p . For the proof of part (ii) we proceed 
analogously by exploiting fl2TT5|) and fETTTj) instead of f l27L4"P and < HJ6\i . □ 

From Proposition 13.41 we have along the line of Corollary 13.31 the following 



Corollary 3.5. Let be satisfied the assumptions of Proposition \3^4\ and let a n be 
chosen by ^3. II) . Then, in the both cases (i) and (ii) of Proposition \3~4\ we have 

(i) \\x 5 n -x%<E-(l + M/m), (3.16) 

(ii) \\x s n - x ] || p < 2E ■ M/m. (3.17) 

3.3. Error bounds in X. For deriving order optimal error bounds for the total 
error \\x 5 n — x* || with a n chosen a priori by (13. ip we employ interpolation techniques 
from Proposition 12.71 and use the results of Corollary 13.51 which provides a bound 
for \\x 5 n — x^Wp and the results of Corollary 13.31 which together with the first link 
condition Al(i) provides a bound for \\p{G)(x s n — a^)||. 

Theorem 3.6. Let be satisfied the assumptions of Proposition l3~l\ and 3.4 and let 

a n be chosen a priori by $3. If the function £ p (t) := tp 2 ^ 1 ^ 2 ^) is convex, then 
x 5 n is order optimal on the set M P) e in the both cases of high order regularization 
s > p and low order regularization s < p. In fact, in both cases, 

\\x 5 n -x^\\<c 1 [% 1 {c 2 5)} p (3.18) 

with some constants c\ and c 2 which can be extracted from the proof. 

Proof. Due to Corollary 13. 5[ Corollary 13.31 and Assumption Al(i), in both cases 
of high- and low order regularization we have 

\\ x n ~~ ^Wp — ^1 an d \\@(G)(x 5 n — x^)\\ < k 2 5 

with some constants k\ and k 2 . Using the interpolation estimate (I2.19P of Propo- 
sition O yields fl3~T8|) . □ 
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Note that for the finitely smoothing case of Example 12 . 61 we have £ p (£) = t( a+p )/ p , 
which is convex for arbitrary p > 0. 

3.4. Revisiting the low order case. The error bounds given in Subsection 3.3 
require in both cases of high order and low order regularization the both link 
conditions Al(i) and Al(ii). We will show in this subsection that in the case of 
low order regularization s < p order optimal error bounds can be obtained without 
the second link condition Al(ii). However, this will only be possible for s > 0. 
We exploit in our study the property 

VXg n (X) < ^ for AG (0, ||T|| 2 ] (3.19) 

which follows from the both properties (12. 2p and (12. 3p of Proposition 12. II and start 
by providing some error bound in the || • || s - norm for arbitrary a n > 0. 

Proposition 3.7. Let x 5 n be defined by 112. h be defined by 112. 13\) and assume 
the solution smoothness A2. If h 2 is operator monotone and h(t)/t is decreasing, 
then under the link condition Al(i), 

\\x 5 n -x ] \\ s <5^ + Eh(a- l /m 2 ). (3.20) 

Proof. For estimating \\x s n — x n \\ s in the low order case s < p we use the error 
representation (12.61) and obtain due to v^Afi'a(A) < yfdn, see ( I3.19p . the estimate 

\\x s n -x n \\.= \\g n (T*T)T*(y s -y)\\<6yfr. (3.21) 

For estimating \\x n — x^ \\ s we use the error representation (12.61) . exploit the estimate 
( I2.15P of Proposition 12.51 and obtain due to \\x^ — Xq\\ p < E the estimate 

t _ rr. II — II- (T*T\n~ S (^ 



\x' — x 



r n (T*T)G- s (x^ - x ) II < E \\r n (T*T)h (T*T/m 2 ) \\ . (3.22) 

For estimating the norm term in (I3.22p we distinguish two cases A < cr^ 1 and 
A > a^ 1 . In the first case A < a^ 1 we use r n (A) < 1, or equivalently, g n (X) > 
exploit the increasing behavior of h(t) which is always satisfied since h 2 is operator 
monotone and obtain 

r n {X)h{X/m 2 ) < h{X/m 2 ) < h{a^/m 2 ). 

In the second case A > a^ 1 we use Ar„(A) < cr^ 1 , exploit the decreasing behavior 
of h(t) 1 1 and obtain 

r n (X)h(X/m 2 ) < a7 n l h(X/m 2 )/X < h^/m 2 ). 

From the both cases we obtain that ( I3.22p attains the form \\x*— x n || s < Eh(a~ l / m 2 ) 
From this estimate and (13.2 ip we obtain ( I3.20p . □ 

Since the parameter choice (13. ip can be written in the equivalent form = 
Eh{o~ x /m 2 ) we obtain from Proposition 13.71 the following 

Corollary 3.8. Let be satisfied the assumptions of Proposition \37K and let a n be 
chosen a priori by /13.1\) . Then, 



t-xm<2E 



5 
mE 



p—s 



(3.23) 
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Proof. From (12. 121) we have g(\) = A s ip s (^) and £?(A) = A p ip p (\). Consequently, 

i> s {\) = A S -%(A) and ^ P (A) = A P ~%(A). (3.24) 

We use the first equation of (I3.24p . substitute A = ip' 1 ^) and obtain ip s (ip~ l it)) = 
ttyp 1 (t)] s ~ p . From this equation we conclude that the a priori parameter choice 

(EH]), which is equivalent to cr^ 1/2 /m = ^ [ip~ 1 (^)j 

an 1/2 /m = ip s (ip' 1 (^)), or equivalently, 

-1/2 



s—p 



can be rewritten as 



m 



5 



mi? 



(3.25) 



Clearly, ( 13.251) is equivalent to ip p yip s 1 {^n 1 ^ 2 / m )) = ^f- We use the second 
equation of ( I3.24p and write this equation in the form 



ft 



a. 



-1/2' 



m 



p- s -1/2 



5 



m 



mi? 



From this equation and the definition of /i by ( 12.131) we see that the parameter 
choice (13. ip can be written in the equivalent form S^/On = Eh[a~ x /m 2 ). Due to 
this equation, estimate (I3.20p of Proposition 13.71 attains the form 

\\x 5 n -x^\\ s <26y/a^. (3.26) 

From this estimate and the parameter choice (13.11) we obtain (I3.23p . □ 

Now, by using the both estimates (I3.23P and (13. 7ft . we obtain the following order 
optimality result for x 5 n on the set M Pj e- 

Theorem 3.9. Let x 5 n be defined by \2. 1\) with a n chosen by \3. 1\) . assume the 
link condition Al(i) and the solution smoothness A2. If h 2 is operator monotone, 
h(t)/y/i is decreasing and £ s (£) := ip 2 (t 1 ^ 2 ^) is convex, then 



\x„ 



x 



+ 11 < 2E 



5 



mE 



(3.27) 



Proof. From estimate (13 .7ft and Assumption Al(i) we have the estimate 

\\g(G)(x s n -x j! )\\ <28/m. (3.28) 

We apply the interpolation estimate ( I2.19P of Proposition 12 . 71 and obtain together 
with (jS/jgD and (j3?28]) the estimate 

\\x 5 n - x+|| < 25a]/ 2 [C 1 K 172 /™)] 8 ■ (3-29) 

It remains to show that for the parameter choice ( 13. ip both right hand sides of 
(I3.27P and ( I3.29P coincide. We use formula (3.25) and obtain that (I3.29P can be 
written in the equivalent form 

5 



rl-^\\<25o l J 2 



-1 



mE 



(3.30) 



1 /2 

Now we rewrite (3.1) as a„ 
obtain (13727]) . 



7 i^p 1 (mE)Y > substitute this into ( 13.301) and 

□ 
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4. Discrepancy principle 

cho: 
<-p) 

may be used where C\ 



If the constants m and E in the a priori parameter choice (13. ip are unknown 
then the parameter choice a n — [ ^- J ip p [ 



and C2 are positive constants guessing m and -E 1 , respectively. For this parameter 
choice, the order optimality results of Theorems 13.61 and 13.91 still hold true. In 
case of rough estimates for m and E, and in particular in cases where p and ip p are 
unknown, a posteriori rules for choosing o~ n have to be used. In the discrepancy 
principle (see |24j ) the regularization parameter o n is chosen as the solution of 
the nonlinear equation 

d{a n ) := \\Ax 5 n - y s \\ = C5 (4.1) 

with some constant C > 1. For practical reasons it makes sense to choose a n such 
that 

Cx5 < d(a n ) < C 2 6 (4.2) 

with some constants C%, G% that obey 1 < C\ < Ci. In computations it makes 
sense to choose Ci with Ci > C\. 

Remark 4.1. For realizing the discrepancy principle (14. ip or (14.21) approximately, 
one practical way is as follows. We start with some large a\ in (II. 2p . use a 
decreasing a-sequence and iterate as long as the discrepancy is in the magnitude of 
the noise level. More accurately, we consider the decreasing sequence A = {a^lfcLi 
and choose n as the first integer for which 

\\Ax 5 n -y 6 \\<C5 <\\Ax s k -y 5 \\, 0<k<n (4.3) 

with some C > 1. Some care is required for the final iteration step in which one 
has to take care that the discrepancy becomes not too small and remains in the 
magnitude of 5. This can be guaranteed by assuming that the final a n is not too 
small and obeys 

l/a n < can-! (4.4) 

with some positive constant c. For the geometric sequence A = {q h ~ 1 ai}'^ =1 with 
some q < 1, assumption (I4.4p is satisfied with c = 1/q, see [7]. We show in 
Subsection 4.3 that for the version (14. 3 p of the discrepancy principle, analogous 
convergence rate results to that of the a posteriori rule (14.21) hold true. 

4.1. Properties. Due to (12.11) . the discrepancy y 5 — Ax 5 n can be represented by 

y 5 - Ax s n = r n (TT*)(y 5 - Ax ) = (j[a k (TT* + a k I)-^j (y s - Ax ). (4.5) 

From this representation we conclude that the discrepancy is monotonically de- 
creasing with respect to the iteration number, that is, 

\\y 5 -Ax{\\ < \\y 5 -Axl_ x l A; = 1,2,... 

For a n — > oo we have r n (A) — > 0, and for o n — > we have r n (A) — > 1. Therefore, 
by (14. 5p . we have the two limit relations 

lim \\y s - Ax s n \\ = and lim \\y s - Ax 5 n \\ = \\y 5 - Ax Q \\. 

(J n y OO (7 n -s>0 
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From both limit relations we conclude that under the condition \\y s — Axq\\ > 
C5 there exists o~ n (not necessarily unique) that obeys rule (14. ip or rule ( 14.31) . 
respectively, and that under the condition \\y s — Axq\\ > C2S there exists a n that 
obeys rule (14. 2D . 

Now we assume that for some given cr n -i we have || J 4n^_ 1 — > C5. Then, the 
discrepancy d(a n ) := || Ac* — y s \\ as a function of a n possesses following properties: 

(i) For a n — > we have the limit relation \im an ^ d(a n ) = 0. 

(ii) For a n — > 00 we have lim^^oo d(at n ) = WAx^-i — y s \\ > C5. 

(iii) The function d(a n ) is continuous and strictly monotonically increasing. 

As a consequence, there exists a* with c?(a*) = C5. 

Following proposition gives us some monotonicity property for the error \\x 5 n — 
x'\\ s with respect to the X s -norm which tells us that the iteration (11.21) should 
not be stopped as long as \\Ax 5 n — y s \\ > 6 holds. In the special case s — 0, such 
monotonicity property may be found in [5]. 

Proposition 4.2. Let x^ E X s , let x s n be defined by the iteration ( fi.ffj) and let 

\\Ax 5 n — y s \\ > 5. Then, 

\\x 5 n -x% < Hx^-^ll,. (4.6) 
Proof. The iteration (jl.2p can be rewritten as 

x 5 n = x 5 n _ 1 + B- 2s A*z n _ 1 with z n „ 1 = {TT* + a n I)- 1 {y 5 -Ax 5 n _ x ). 
Consequently, for d := — a^H 2 . — — a^H 2 . we have 

3* IL ll^ra— 1 lis 

= (2x*_! - 2x f + B- 2s A*z n . 1 , B- 2s A*z n ^) s 

— (%n— 1 + ~~ 2x^, i? A 2 n _i) s 

= + Ax s n - 2y, z n _i) 

= {2(y 5 -y) + (Ac n _ x - y 5 ) + (Ac* - /), z n _ x ) 

((y a -AO + (y a -A4),z n _!) 



2||z, 



n-l 



Let r n := y* — Ax s n . Then, from (14.5ft we have r n = a n (TT* + a n I) 1 r n -\. Hence, 
the element z n _i can be written as z n -\ = a~ l r n . Consequently, 

11 $ tn2 11 <5 + 1 1 2 2 r„ / (r n _i + r n , r n ) \ „> 

K - ^ lis - Fn-1 - ^lls < — 5 ^|T^ ■ (4-7) 

«n \ *lr n|| / 

We use again the identity r n = a n (TT* + a n /)~ 1 r„_ 1 , or equivalently, r n _i = 
a~ 1 (TT* + a n I)r n , multiply by r n and obtain 



„_i,r n ) = a^HTVJ 2 + ||r„|| 2 > ||r n || 2 . (4.8) 
From (|4~77j) . (fl~%j) and ||r„|| > <5 we obtain \\x 5 n -x ] \\ 2 s - \\x s n _ x -x ] \\ 2 s < 0. □ 
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4.2. Error bounds in X. In this subsection we show that for o n chosen by 
(14. ip or (I4.2j) . respectively, the order optimal error bound ( 13. 18ft holds true under 
analogous assumptions of Theorem 13.61 In a first proposition we provide some 
estimate for the regularization parameter a n chosen by (14.2ft . 

Proposition 4.3. Let x s n be defined by \2. II) . h and w be defined by $2.1 Si) . o~ n 

be chosen by the discrepancy principle QJfy with 1 < C\ < C2 and assume the 
solution smoothness A2. 

(i) High order regularization (s > p): If w 2 := 1/h 2 is operator monotone, 
then under the link condition Al(ii), 

(d - 1)6 < E^h{a- X /M 2 ). (4.9) 

(ii) Low order regularization (s < p): If h 2 is operator monotone and if 
h(t)/y/i is decreasing, then under the link condition Al(i), 

(d - 1)5 < Ey^hia-'/m 2 ). (4.10) 







Proof. Let us prove part (i). From ( 12.11) we have y — Ax n = r n (TT*)(y — Ax. 
Due to rule (|4.2|) . the identity y — Ax n = r n (TT*)(y — Axq) and the estimate 
||r n (TT*)|| < 1 we obtain that 

C x & < \\r n (TT*)(y s - Ax )\\ 

< \\r n (TT*)(y - Ax )\\ + \\r n (TT*)(y - y 5 )\\ 

< \\y-Ax n \\+5. (4.11) 
From the proof of Proposition 13.11 we have that 



\\y-Ax n \\ <E^h{a- l /M 2 ). 

This estimate and (I4.1ip provide the desired estimate (14. 9 j) and the proof of part 
(i) is complete. For the proof of part (ii) we proceed in an analogous way, but use 
instead of (I2.14p the estimate (I2.15P which requires the link condition Al(i) and 
the operator monotonicity of the function h 2 . □ 

From Propositions 13.41 and 14.31 we obtain that the total error x s n — x^ is bounded 
in the || • || p - norm for the a posteriori parameter choice a n chosen by the discrep- 
ancy principle (14. 2p . 

Proposition 4.4. Let x 5 n be defined by \2. o~ n be chosen by the discrepancy 
principle \4-2\j with 1 < C\ < C2 and assume the solution smoothness A2 and the 
link condition Al. 

(i) High order regularization (s > p): If w 2 := 1/h 2 is operator monotone, 

II £ til E M n M ,* S 

4 -x%< • — + E ■ -. (4.12) 

Gi — 1 m m 

(ii) Low order regularization (s < p): If h 2 is operator monotone and h(t)/y/i 
is decreasing, then, 

U _ X% <Z.E +E .E. (4,3) 
61 — 1 m m 
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Proof. In the case (i) we exploit the increasing behavior of \rth(t) and conclude 
from cr~ l /M 2 < cr^/m 2 that hia^jM 2 ) < ^h^a^/m 2 ), which together with 
part (i) of the two Propositions 13.41 and 14.31 provides (I4.12p . In the case (ii) we 
exploit the decreasing behavior of h(i)/y/i, or equivalently the increasing behavior 
of y/iw(t) and conclude from or^/M 2 < a^/m 2 that w(a~ 1 /M 2 ) < ^w{o~ x /m 2 ), 
which together with part (ii) of the two Propositions 13.41 and 14.31 provides ( I4.13p . 

□ 

Now we are in a position to prove the main result of this section. In our next 
theorem we will see that order optimal error bounds can be guaranteed in case o~ n 
is chosen by the discrepancy principle ( 14. 2 h with 1 < C\ < C 2 . 



Theorem 4.5. Let be satisfied the assumptions of Proposition \4^4\ and assume in 
addition that f p (t) := ^(t 1/(2p) ) is convex. Then, 

\\x 5 n - x^\\ < Cl (c 2 6)] p (4.14) 

with some constants C\, c 2 which can be extracted from the proof. 

Proof. Due to Proposition 14. A\ in both cases (i) and (ii) of high- and low order 
regularization the total error obeys 

\\x{-x ] \\ p <cE (4.15) 

with some c > 1 and a n chosen by the discrepancy principle (14. 2 j) with 1 < C\ < 
C 2 . From (14. 2 p and the triangle inequality we have 

\\Ax 5 n - || < || Ax* - y s \\ + \\y - y 5 \\ < (C 2 + 1)5. 

Using in addition the link condition Al(i) yields 

\\p(G)(x s n -J)\\<(C 2 + l)8/m. (4.16) 

Now the result of the theorem follows from (I4.15p . (I4.16P and Proposition 12.71 □ 

4.3. Error bounds for rule f 14 . 3 f) . For the a posteriori rule (14. 3 P of choosing 
the regularization parametr a n , analogous order optimal error bounds to that of 
Theorem 14.51 can be obtained. 

Theorem 4.6. Let x 5 n be defined by Ii2. let a n be chosen by rule 1^4 -3\ ) where ot n 
obeys let the both Assumptions Al and A2 hold and assume that £ p (t) := 

(t 1 ^ 2 ^) is convex. Assume further 

(i) in case of high order regularization (s > p) that w 2 := 1/h 2 is operator 
monotone and 

(ii) in case of low order regularization (s < p) that h 2 is operator monotone 
and h{t)/yt is decreasing. 

Then, the regularized solution x 6 n obeys the order optimal error bound 

Wxi-x^W <c 1 [^ 1 (c 2 6)] p (4.17) 
with some constants c\, c 2 which can be extracted from the proof. 
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Proof. We give the proof for the high order case s > p, the proof for the low order 
case s < p is similar. In the first step of our proof we proceed according to the 
proof of Proposition 14.31 exploit that C5 < \\r n -i(TT*)(y s — Axq)\\ and obtain 



(C-ljK^Aj^/M 2 ). 
From this estimate and (13 . 1 0[) we obtain 



»n-« t L<T? V ; + E ■ Mm. 4.18 

Now we consider two cases. In the first case with a~\/M 2 < a^ 1 jm 2 we use the 
increasing behavior of \fth(t) and obtain from (I4.18P the estimate 

\\<-x%<-^ l ~ + E.M/m. 

In the second case with a~\jM 2 > o-^/m 2 we use the decreasing behavior of h, 
exploit in addition that due to (14. 4p we have a n = l/a n + er n _i < (c + l)cr n _i, or 
equivalently, cr^-i ^ ( c + l) "^ 1 ' an( i obtain from ( I4.18P the estimate 

E 

— ar+Hp < - — - • Vc + 1 + E ■ M/m. 

From the both cases we have that II cc^ j | ip can be estimated by 

E ( 

\\x 5 n ~ ^\\ p < - — - • max |M/m, y/c + lj + E ■ M/m. (4.19) 

In the second step we proceed according to the proof of (I4.16P and obtain 

\\g(G)(x s n -x^\\<(C + l)6/m. (4.20) 

In the final third step of the proof we use the both estimates ( I4.19P and f )4.20p . 
apply Proposition 12.71 and obtain (I4.17|) . □ 

4.4. Discrepancy principle revisited. The error bounds given in Subsection 
4.2 require in both cases of high order and low order regularization the both 
link conditions Al(i) and Al(ii), and the assumption C\ > 1 in the discrepancy 
principle (14. 2p . We will show in this subsection that in the case of low order 
regularization s < p order optimal error bounds can be obtained without the 
second link condition Al(ii). Our estimate in Theorem 14.81 shows that C\ = C<i = 1 
in the discrepancy principle (14.21) is best possible in the sense of minimal error 
bounds. We start our study with some important inequality. 

Proposition 4.7. For < s < p, the regularized solution x 5 n defined by i2. 1\) 
obeys the estimate 

Uxi-y^ + a-'Wxi-xm < ^Hry^T^G-^xt-xo)!! 2 

+ lly-/ll 2 - (4-21) 



IMPLICIT ITERATION METHODS IN HILBERT SCALES 19 

Proof. Let A : X s — > Y be the restriction of A to X s C X and A* : F — >• X s its 
adjoint. Due to the valid identity (Ax,y) = (x,A*y) s = (x,A*y) = (x,G 2s A*y) s 
for all x G X s and y G F we conclude that the adjoint A* : F — > X s of the 
operator A : X s -» F is given by A* = G 2s A*. The operator A*A : I s 4 I s is 
self-adjoint. Further, there holds 

G s g n (T*T) = g n (A* s A)G s . (4.22) 

Consequently, the regularized solution ( 12.11) which is an element of the space X s 
can be written in the equivalent form 

x s n -x = g n (A* s A)A* s (y 5 - Ax ). 

From the valid identity x s n — x 1 " = — r n (A*A)(x^ — x ) + g n {A*A)A*{y & — Ax^) and 
the identity g n (A*A)A* = A*g n (AA*) we obtain 

= \\r n (AtA)(x^-x )\\l + \\g n (A:A)A* s (y s -Ax^ s 

-2(Ag n (A* s A)r n (A* s A)(xi - x ),y s - Ax*). (4.23) 
We introduce the abbreviations 

R n := g n (AA* s )r n (AA* s ) and y s := / - Ax , 
decompose y$ into the sum A(x^ — xq) plus y s — Ax^ and obtain the equality 
(R n y 5 ,y 5 ) = {R n A{x ] -x ),A(x^ - x )) + (R n (Ax^ -y 5 ),Ax^ - y 5 ) 

+ 2{R n A{x^ -x ),y s -Ax ] ). (4.24) 
Addition of the equations (I4.23P and f)4.24p yields 

(R n y 5 ,y 5 ) + \\x s n - xt\\ 2 . = \\r n (A* s A)(^ - x )\\ 2 s + \\g n (A:A)A* s (y s - A^)\\ 2 

+ (A* s R n A(x^ - x Q ), (x f - x )) a 
+ (R n (Ax^-y s ),Ax^-y s ). 

We use the valid identities 

r 2 n (A;A) + A* s R n A = r n (A* s A), Ag 2 n (A* s A)A* s + R n = g n (AA* s ) 
and obtain from the above equation 

(R n y s ,y s ) + \\x s n -xm 2 s = (r n (A* s A)(x^ - x ), x f - x ) s 

+ (g n (AA*)(Ax* - y s ), Ax* - y 5 ). (4.25) 
By exploiting properties (i) and (iv) of Proposition I2.ll we obtain 

(a) a-\g n {AA* s ){Ax^-y s ),Ax^-y s ) < \\Ax^-y s f, 

(b) a-\R n yiyi) > {r 2 n (AA:)y s ,y s ) = \\r n (AA* s )y s \\ 2 = \\Ax 5 n - y s \\ 2 . 
We multiply (I4.25P by cr" 1 , use the estimates (a) and (b) and obtain 

WAxi-y^+a^Wxi-x^Wl < \\Ax^ -y & \\ 2 +a- l {r n {A* s A){x^ -x Q ), x^-x ) s . (4.26) 
Finally we observe that due to ( 14.221) we have 

(r n (A* s A)(xi - x ), xt - x ) s = \\r l J 2 (T*T)G- s ^ - x )|| 2 . 
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From this identity and ( 14.261) we obtain (14.211) . 



□ 



From Proposition 14.71 and Proposition 12.81 we obtain the main result of this 
subsection. 

Theorem 4.8. Let x s n be defined by Ii2.1\) and a n be chosen by the discrepancy 
principle ^4-2ty with 1 < G\ < G 2 , assume the link condition Al(i), the solution 
smoothness A2 and < s < p. If f defined by Ii2.20\) is convex, and : = 

i^lit 1 ^ 2 ^) is convex where ip s is given by Ii2.12\) , then 



x , 



x T l| < E 



x f( C 2 + l)6 
mE 



(4.27) 



Proof. For a n chosen by the discrepancy principle (14.21) the estimate (I4.2ip of 
Proposition 14.71 attains the form 



CfS 2 + a, 



X < * + °?\\r!/*(7~T)G 



- s ^-x )|| 2 . 



Since d > 1, we have \\x s n - xt||2 < \\rl /2 (T*T)G- s (x^ - x )|| 2 . We use the 
representation G^ s (x' f — x n ) = r n {T*T)G~ s (x^ — xq), see ( 12.61) . use Assumption 
A2 and obtain 



\x" — x 



f ||* < \\rll 2 (T*T)G- s (x^ 



(GP- 2s (x1-x n ),G-P(x^-x )) 



< E\\Xr. 



X' 2 



s— pi 



(4.28) 



where x n is the regularized solution with exact data. For estimating \\x n — a;t|| 2s _ p) 
we use estimate ( 12.21R of Proposition 12.81 and obtain 



< E 



c 1 



\g(G)(x r 



x 1 



E 



(4.29) 
Ax ) = 



For estimating ||Ar n — Aa;t||, we use CI 2 . 8 j) . the identity r n (TT*)(y 5 
y s — Axft, r n (X) < 1 and ( 14. 2 p and obtain the estimate 

\\Ax n -Axm = \\r n (TT*)(y - Ax )\\ 

< \\r n (TT*)(y s - Ax )\\ + \\r n (TT*)(y - y 5 )\\ 

< (C 2 + l)5. 

Hence, by using Al(i) we have \\g{G)(x n — x*)\\ < (C 2 + 1)5 /m. Since ip^ 1 is 
monotone, we obtain from (I4.29P the estimate 



< E 



! {{ C 2 + l)6 
mE 



(4.30) 



Next, let us estimate ||^(G)(x* — 



\\Ax 5 n - Act || < \\Ax s r 



'• T ' ,N Using Assumption Al(i) and the estimate 
i + lly-^ll < (C 2 + 1)5 yields 



\g(G){x s n -x*)\\ < (C 2 + l)S/m. 



(4.31) 
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Now we apply the interpolation estimate ( 12. 191) of Proposition 12.71 and obtain by 
using ( jOQD , (IOT]l and the abbreviation ^ := (C ^+ 1)g that 

114-^11 <£[V>;W~ S - [^(W^i)]*"*)]'- (4-32) 
From the first equation of (I3.24p we have f/'.r 1 (^(A) • A s ~ p ) = A. Substituting 
A = ip- 1 (8i) yields tp; 1 1 (5i)] s_p ) = ^(^l)- From this equation and (1Q2]1 

we obtain OMU). □ 

5. Practical implementation 

For the practical application of implicit iteration methods in Hilbert scales one 
has to make different decisions: First, one has to choose the operator B, second, 
one has to fix the number s in the method (II. 2p . third, one has to choose the 
starting value xq and to fix the numbers a k , k = l,...,n, and fourth, one has 
effectively to realize the discrepancy principle ( 14. II) with a little number n of 
iteration steps. The choice of B and Xq should be done in dependence on the 
expected smoothness of the element x< — Xq such that Assumption A2 holds true 
for p sufficiently large, and s should have the magnitude of p. In our further 
study we concentrate on the choice of the numbers a k , k = 1, ...,n for effectively 
realizing the discrepancy principle (14.11) or (14. 2|) or (14.31) . respectively, with a little 
number n of iteration steps. In a first proposition we give an upper bound for 
the regularization parameter of the discrepancy principle in case n = 1 which 
will serve us as starting value for the iteration (1.2). To our best knowledge, so 
far there have not been upper bounds for the regularization parameter of the 
discrepancy principle in the literature. 

Proposition 5.1. Let n = 1, let xf the regularized solution Ii2. 1\) and let a\= old 

be chosen by the discrepancy principle ( [^. 1\ ) with C > 1. // \\y 5 — Axo|| > C5, 
then 

C5\\G s A*(y s -Ax )f 
a ° (\\yS-Ax \\-C5)\\y*-Ax \\ 2 - { ' } 

Proof. Let x s a = xq — (A* A + a>G~ 2s )~ l A* (Ax — y 5 ) and a = old be the regulariza- 
tion parameter that obeys the discrepancy principle \Ax & a — y 5 \\ = C5. For solving 
this nonlinear equation, Newton's method applied to the equivalent equation 

g( r ) = \\Ax 5 1/r -y 5 \\- 1 -(C5)- 1 = (5.2) 

is studied in [16] which results in the iteration 



r k +i = r k r (5.3) 



where vf, is given by v± , = (A* A + r^ 1 G^ 2s )^ 1 G^ 2s (xy r — x ). From [161 Theo- 
rem 3.5] we know that the iteration ( 15.31) possesses the following properties: 

(i) The sequence (r^) converges globally and monotonically from the left to 
r£> for any starting values < ro < rp := 1/old- 

(ii) The speed of convergence is locally quadratic. 
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For r — > 0, the both limit relations 



A/r = *o and lim r k 3 (v 5 1/r , G- 2s (x[ /r - x )) = \\G s A*(y s - Ax ) 



are valid. We execute one iteration step of the iteration (15. 3p with starting value 
r = and obtain due to the above limit relations that 

= (\\y s - Ax \\ - C6) \\y s -Ax \\ 2 
Tl C6\\G*A*(y s -Ax )\\ 2 

Due to the above property (i) we have r\ < rr>. Since r and a are related by 
a = 1/r we obtain ( 15. ip . □ 

Based on the Newton iteration ( 15. 3 h we propose following strategy for effectively 
realizing the discrepancy principle (14. 3 j) with a little number n of iteration steps. 

Algorithm 1 Global convergent Newton iteration for rule (14. 3p 



Start with initial data y s , A, G, s, 5, C := 1.1 and xq. 
if \\Axq -y s \\ > C5 then 

Compute a by the right hand side of (15. ip with C = 1. 
Compute x := x — (A* A + aG~ 2s y l A* (Ax — y 5 ) and set n := 1. 
while ||Ar — y s \\ > C5 do 

Compute v := (A* A + aG- 2s y l G^ \x - x Q ). 
TT , 1 \\Ax - y*]]- 1 - 6- 1 

J • -p ^ QiXj G f ' — — ■ — j 

a a 3 (v, G~ 2s (x — Xq)) \\Ax — y 5 ^ 3 ' 



8 
9 
10 



x ■= x, a := 1/r, x := x - (A* A + aG' 28 )- 1 A* (Ax - y 5 ) 
end while 
end if 



For discussing some properties of Algorithm 1, we will work with the notation 

:= 4-i - A + aB 2s )~ 1 A*(Ax s k _ l -y s ), k = 1, 2, 

that indicates the dependence of x s k defined by (II. 2p on the parameter a. We start 
by some monotonicity property of the sequence (afe)^ =1 in the iteration (11.21) . 

Proposition 5.2. The regularized solutions x s k , k = l,...,n, obtained by Algo- 
rithm 1 have the form U.fy) . The related sequence ((Xk)k=i ^ s strictly monotonically 
decreasing. 

Proof. In steps 3 and 4 of Algorithm 1, a\ and x{ = xf(ai) are computed. Then, 
the while loop (steps 5 - 9 of Algorithm 1) is executed n — 1 times to obtain 
and x s k = x k (ak) for k — 2, ...,n. The parameter a = ak := 1/r*. (see step 7 of 
Algorithm 1) is obtained by performing one Newton step for solving the nonlinear 
equation 

g(r) = \\Ax S k _ 1 (l/r)-y 5 \\- 1 -5- 1 = 
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with starting value r k -\ = 1/otk-i- It can be shown (compare |16| ) that the 
function g possesses following properties: 

(i) There hold the two limit relations 

lim g(r) = \\Axt_ 2 — 2/^ II 1 — < and lim g(r) = +oo. 

r— >0+0 ' r— >oo 

(ii) The function g : IR + — > M is monotonically increasing and concave. 
From these properties and g{rk-i) < we conclude that r k > r^-i- It follows that 
ctk < oik-i for all k = 2, . . . , n. □ 

For discussing convergence properties of Algorithm 1 we consider Tikhonov 
regularization 

4(Pm) ■= xo - (A* A + /3 m C7- 2s )-M*(Ax - y 5 ) (5.4) 

and assume 

(i) (3k = t k~ i k = 2, . . . , m, is obtained by the iteration 
r k = r fc _i r , (5.5) 

where v\, r is given by uj, = {A* A + r _1 G ,_2s ) _1 G _2,s (x^ r — x ) and . 
is given by xf, r = x{ (1/r), 
/..n • u (||?/- Ar || -£)||^-Ar || 2 

ii 7"i is chosen as n = n T~i ; — rnr, and the iteration (15.51) 

5 \\G s A*(y 5 - Ax )\\ 2 " k 

is stopped with the first integer m for which, with C := 1.1, 

||Arf(/3 m )-y 5 || < C5 < \\Ax{(f3 k ) - y 5 \\, < k < m. (5.6) 

From |16j we know that the iteration ( 15. 5ft converges globally and monotonically 
from the left to the solution of the equation g(r) = \\Ax{(l/r) — — S^ 1 = 0, 

and that in the vicinity of the solution we have quadratic speed of convergence. 
It follows that by the stopping rule (15.61) a finite number m of iteration steps is 
defined. Our next proposition tells us that Algorithm 1 is not slower than the 
iteration (j5.5p with stopping rule (15. 6p . 

Proposition 5.3. Letm be the number of iterations of method ( 15. 5\) with stopping 
rule A5.6\) . Then, n <m, where n is the number of iterations of Algorithm 1. 

Proof. Assume that a,\ and x\(a\) in steps 3 and 4 of Algorithm 1 are computed, 
which coincide with f3\ and xf(/3i) of the iteration (15. 5p . Then, in the first iteration 
step of the while-loop (steps 5 - 9 of Algorithm 1) we obtain a 2 and 

x 5 2 {a 2 ) = xf(ai) - (A* A + a^ 23 )- 1 A* (ArJ(ai) - y 5 ). 

For x 2 (a 2 ) computed in this way we have 

y 5 - Ax 5 2 (a 2 ) = a 2 (TT* + a 2 I)~ 1 ai (TT* + aj)' 1 ^ 5 - Ax ). (5.7) 

On the other hand, from the iteration (15.51) we obtain after the first step the 
regularization parameter 2 = cx 2 and the regularized solution x{((3 2 ) which obeys 

y s - Ax(((3 2 ) = (3 2 (TT* + (3 2 )-\y 5 - Ax ). (5.8) 
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Comparing both identities ( 15. 71) and (15.81) and observing that «2 = P2 we obtain 
that \\y 5 — Ax 2(0(2) \\ < \\y 5 — 111 a similar way we obtain that 

\\y s - Ax s k (a k )\\ < \\y 5 - Ax{(/3 k )\\, k = 3,...,n, 

where (a k ) is the sequence generated by Algorithm 1 and (fik) is the sequence 
generated by ( 15. 51) . From this estimate we obtain that Algorithm 1 terminates 
not later than the iteration ( 15. 5ft with stopping rule (15.61) . □ 

After termination of Algorithm 1, different cases can appear: 

(1) We have 5 < \\Ax 8 n — y s \\ < C5 with C = 1.1. In this case, all three 
Theorems 14.51 14.61 and 14.81 apply. 

(2) We have WAx^ — y s \\ = 5. Then, both Theorems 14.61 and 14.81 apply. 

(3) We have \\Ax s n — y s \\ < 5. In this case, Theorem 14.61 applies. 

Our next proposition tells us that in all three termination cases (1) - (3), the 
additional assumption (14. 4p of Theorem 14.61 is satisfied with some c < 1. 

Proposition 5.4. The regularized solution x s n obtained by Algorithm 1 has the 
form \2. 1\) with some sequence (a k ) k=1 that obeys assumption (f^.^| ) with c < 1. 

Proof. Consider the final iteration of the while-loop (steps 5 - 9 of Algorithm 1). 
Starting from x = a;^_ 1 (o; ri _i) with \\Ax — y s \\ > CS, a n := l/r n is obtained by 
performing one Newton step for solving the nonlinear equation 

g(r) = \\Ax 5 n _ 1 (l/r)-y 5 \\- l -5- 1 = 

with starting value r n _i = l/a n _i. As a result, we obtain some a n < a n _i, see 
Proposition 15.21 and the final regularized solution x 5 n is obtained by 

<K) = 4-iK-i) - {A* A + a n G~ 2s )~ 1 A*(Ax 5 n _ 1 (a n ^i) - y 5 ). 

Some formal computations show that x 5 n (a n ) can be rewritten as 

x 5 n (a n ) = x 5 n _ x (a n ) - {A* A + a t ^ l G- 2 ')- 1 A*(Ax s n _ 1 (a n ) - y s ). 

Since the function g is monotonically increasing and concave and since g(r n _i) < 
we conclude that the element obeys || Ac^-^ctn)— y s \\ > 5. It follows that 

the final two parameters a n _i and a n in the iteration (II. 2p can be interchanged 
such that we have a n > a n _i. This yields (14.4ft with some constant c < 1. □ 

6. Numerical experiments 

In this section we perform numerical experiments for computing regularized 
solutions by Algorithm 1. We consider Fredholm integral equations 

[Ax](s) := ! K(s,t)x(t)dt = y(s), < s < 1, A : L 2 (0, 1) ->■ L 2 (0, 1) (6.1) 
Jo 

and differential operators B:DcI 2 (0,l)-> L 2 (0, 1) of first order defined by 

00 

Bx = k(x, ek)ek with efc(t) = V2 sin (/c7rt) . (6.2) 
fe=i 
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Example 6.1. Our test example (derivB from [8]) is (16. II) with kernel function 

s(l - 1) for s < t 
i(l - s) for s > i. 



K(s,t) 



For this kernel function, Assumption Al is satisfied with m = M = ir~ 2 and 
g(t) = t 2 . We consider three subexamples in which the right hand sides y(s), 
the corresponding solutions x'(t) and the maximal smoothness parameters po for 
which Assumption A2 with xq = holds true for all p € (0,po), are given by 



(i) y(s) = -j^r sin27rs, x f (t) = sin 2nt, p = oo, 

(ii) y(s) = | (1 - 2s 2 + s 3 ) , xt( t )=4t(l-t), po = §, 
fiii) 



y(s) = f (1 - s 2 ) , xt(t)=t, po = |. 

The discretization of (16. ip has been done by Galerkin approximation as outlined, 
e.g., in [HUB], guaranteeing that ||a;t|| 2 ~ ||x t (t)|| I/ 2 (0>1 ) and ||y|| 2 w ||y(s) ||z^(o t i) 
holds. As a discrete approximation of the first order differential operator (16. 2p we 
use the (m,m) - matrix 



B:=Bl /2 with B< ( " M 1 



/ 2 -1 \ 

12 -1 



'2 vvlLli ^2 — 2~ 



'•• '•■ -1 

V "12/ 



(6.3) 



compare pj6]. For modeling noise in the discretized right hand side y G M m , for 
given nonnegative a we compute 

s Wvh 
\\ e h 

where e = (ej) is a random vector with ~ A/"(0, 1). In this way of modeling 
noise we guarantee that for the relative error we have \\y — j/* 5 1 1 2/ )| 2/ 1 1 2 = cr. The 
noise level 5 is then given by 5 = o Tables 1-3 show our numerical results 

with xq = 0, where the letter codes in the leftmost column refer to following three 
iteration methods: 

• (TI/DP): This is the method of Tikhonov regularization (15. 4p with xq = 
and s — 1, where the regularization parameter obeys (I5.6P and is obtained 
by the iteration (I5.5P which converges globally and locally quadratically. 

• (IIM/A1): This is the implicit iteration method (11.21) with Xq = and 
s — 1, where the sequence (0^)^=1 i s obtained by Algorithm 1. 

• (IIM/GS): This is the implicit iteration method (11. 2p with xq = and 
s — 1, where the sequence (a^)^ is the geometric sequence (q ,fc ~ 1 ai)fc =1 
with q = 5 as proposed in [7] and stopping rule (14. 3 p with C = 1.1. 

For all three iteration methods our tables contain 

(i) the number n of required iterations, 

(ii) the final regularization parameter a n , 
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(iii) the discrepancy d n := ||Arf(a! n ) — y 6 \\2 of the final approximation for the 
iteration method (TI/DP), and the discrepancy d n := \\Ax 6 n — y S \\2 of the 
final approximation for the iteration methods (IIM/Al) and (IIM/GS), 

(iv) the error e n := \\x{ (a n ) — a;t|| 2 of the final approximation for the iteration 
method (TI/DP), and the error e n := — ajt|| 2 of the final approximation 
for the iteration methods (IIM/Al) and (IIM/GS). 

In our experiments, all three iteration methods have been started first with 

{B- 2 A*y s ,A*y s ) 

a l = 771 — 771 7771 7777 j (6.4) 

(\\y 5 h-S)\\y s \\ 2 2 

compare (15. lh . and second with ati — 1 as done in [7j. In order to keep the 
discretization error small, we have used the dimension number m = 400 in all 
computations. 



Method 


n 
















(TI/DP) 


3 


5.54 E- 


-7 


1.8! 


3 E- 


-4 


3.79 E- 


-3 


(IIM/Al) 


2 


8.85 E- 


-7 


1.71 


3 E- 


-4 


2.94 E- 


-3 


(IIM/GS) 


2 


8.10 E- 


-7 


1.71 


1 E- 


-4 


2.95 E- 


-3 


(TI/DP) 


4 


5.54 E- 


-7 


1.8f 


3 E- 


-4 


3.79 E- 


-3 


(IIM/Al) 


3 


8.85 E- 


-7 


1.7! 


3 E- 


-4 


2.94 E- 


-3 


(IIM/GS) 


17 


1.52 E- 


-5 


1.7! 


3 E- 


-4 


2.83 E- 


-3 



Table 1. Example O (i) with a = 0.01 (5 = a\\y\\ 2 « 1.79 E-4). Top: a x from §6A 
Down: ol\ = 1. 



Method n a n d n e 



(TI/DP) 3 2.75 E-5 7.97 E-4 1.62 E-2 

(IIM/Al) 2 5.14 E-5 7.75 E-4 1.70 E-2 

(IIM/GS) 2 5.18 E-5 7.75 E-4 1.70 E-2 

(TI/DP) 4 2.75 E-5 7.97 E-4 1.62 E-2 

(IIM/Al) 3 5.15 E-5 7.75 E-4 1.70 E-2 

(IIM/GS) 12 4.88 E-4 8.08 E-4 2.46 E-2 

Table 2. Example [64] (ii) with a = 0.01 (5 = a\\y\\ 2 ~ 7.39 E-4). Top: a[ from 

Down: a\ = 1. 



In our numerical experiments we observed that the accuracy of each individual 
regularization method in the three test cases of Examples 16.11 (i) - (iii) is as 
predicted by the theory. In Tables [TJ- [3] we mainly concentrate on the performance 
of the three methods and observe following: 

(1) As far as computational expenses are concerned, the iteration method 
(IIM/Al) performs best. In fact, this method requires the smallest number 
of iterations compared with the other two methods. 
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Method 


?7 




d n 




(TI/DP) 


6 


7.62 E-8 


4.83 E-4 


1.51 E-1 


(IIM/A1) 


5 


1.24 E-7 


4.80 E-4 


1.51 E-1 


(IIM/GS) 


9 


3.98 E-7 


4.95 E-4 


1.62 E-1 


(TI/DP) 


7 


7.62 E-8 


4.83 E-4 


1.51 E-1 


(IIM/A1) 


6 


1.24 E-7 


4.80 E-4 


1.51 E-1 


(IIM/GS) 


22 


4.76 E-7 


5.00 E-4 


1.65 E-1 



Table 3. Example O (iii) with a = 0.01 (S = a\\y\\ 2 ~ 4.60 E-4). Top: ot\ from 



([(HI) . Down: ol\ = 1. 



(2) For the method (IIM/GS), the number of iterations can considerably be 
reduced by starting with a\ from (16.4)) instead of starting with ol\ — \. For 
the other two methods (TI/DP) and (IIM/Al), the number of iterations 
differs only by 1 for the two starting values H6.4[) and <y.\ — 1, respectively. 

(3) In all three iteration methods, the a-sequence is decreasing. How- 
ever, the final regularization parameter a n is smallest for method (TI/DP). 
Comparing the discrepancies for the individual iterations k = 1,2,... 
(which are not contained in the tables) we observed that, for k > 2, d^ in 
method (IIM/Al) is always smaller than c4 in method (TI/DP). 
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